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P**»». ' We consider a time-homogeneous Markov chain X^, n > 0^ valued in R,. Sup- 

^^N ' pose that this chain is transient, that is, X^ generates a cr-finitc renewal mea- 

sure. We prove the key renewal theorem under condition that this chain has 
asymptotically homogeneous at infinity jumps and asymptotically positive drift. 

KEY W^OR-DS: transient Markov chain, renewal kernel, renewal measure, key 

^^ , renewal theorem. Green function. 
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Let ^1, ^2, ■ • • be independent identically distributed random variables with 
a common distribution F on R. Put 5„ = ^i + • • ■ + ^n , Sa = 0, and consider 
the renewal measure generated by sums: 

oc oo 

U{B) = E^I{5„eS} = ^F™(B). 

n=0 n=0 

If F is non-lattice then the celebrated key renewal theorem states that, for 
every fixed /i > 0, 

U{x,x + h] -^ -—— as a: — » oo, (1) 



provided E^i is finite and positive (see, for example. Feller and Orey [5], 
> ; Feller [3 Ch. XI], Woodroofe [25l Appendix]); F is called lattice if it is 

concentrated on some lattice {ka. k £ Z} with a > 0. If i^ is lattice the 
same is true when /i is a multiple of the span a. 
^Sj ■ It is proved in Wang and Woodroofe [23] and in Borovkov and Foss [Sj 

Theorem 2.6] that H]) holds uniformly over certain classes of distributions 
F. Some extensions of the key renewal theorem to the nonidentically dis- 
f~^ I tributed case are considered by Williamson [24], Maejima [18]. Another 

^D ' extension is aimed to include random walks perturbed by both a slowly 

changing sequence and a stationary one, Zn = Sn + rjn + Cn say; see, for 
, , ■ example, Lai and Siegmund [HKTT], Woodroofe [2S], Horvath [11], Zhang 

r> I [IB] , Kim and Woodroofe [T3] . All these extensions deal with perturbations 

jrt ' depending on time rather than on state space; in most cases summands are 

independent and have finite variance. 

Many papers (see, for example, Kesten [12], Athreya, McDonald and 
Ney [2] , Nummelin [20] , Alsmeyer [1] , Kliippelberg and Pergamenchtchikov 
[13], Fuh [S], and also some of their references) are devoted to a Markov 
modulated random walks. Usually a (Harris-) recurrent Markov chain Xn 
is considered with an invariant measure, n say. Conditioned on a realisation 
{x„, n > 0}, one is given a sequence of independent random variables {^„}, 
such that the distribution of ^„ depends only on a;„. Put T„ = ^q + • • • + Cn 
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and assume Tn/n -^ a with probability 1. Then the typical result states 
the convergence 



^^g{Xn,x-Tn) -^ - 7r{dy) g{y,s)ds 

n -' ^— oo 



as X 



for bounded continuous function g satisfying some conditions. The cor- 
responding proofs use probabilistic arguments, notably the construction of 
regeneration epochs for {X^, n > 0} (for example, visit times to some atom). 
This approach eventually reduces the problem to Blackwell's renewal theo- 
rem for sums of independent identically distributed random variables. 

To the best of our knowledge, the only result related to a random walk 
perturbed in state space is due to Heyde [10]. To be more precise, Heyde 
discussed the key renewal theorem for maxima Af„ = max Sk of partial 

0<k<n 

sums: provided E^i is finite and positive 

E^I{A'f„ e (x,a; + /i]} -^ r^— as a; ^ cxj. (2) 



Ee 



1 

It is well known (see, for example, [3 Ch. VI, Sec. 9]) that M„ has the same 
distribution as the reflected random walk on R+ defined by the recursion 

Wn+l = {Wn+^n+l^, Wq^O. 

So, the renewal function generated by the chain Wn has the same asymptotic 
behaviour as described in ^. 

Both the random walk Sn and the reflected random walk on the posi- 
tive half-line Wn are particular examples of Markov chains on R. In the 
present paper we extend the key renewal theorem from these very important 
cases onto asymptotically space-homogeneous Markov chains on R with an 
asymptotically positive drift. Introduce some relevant definitions. 

Let P{x, B), a; G R, _B G B(R), be a transition probability kernel on R; 
hereinafter /B(R) is the Borel cr-algebra on R. Consider a time-homogeneous 
Markov chain X = {Xn, n = 0, 1, 2, . . .} on R with transition probabilities 
P(-, •), that is, 

P{Xn+ieB\Xn^x} = Pix,B). 

Let ^(x) be the random variable distributed as the jump of the chain at 
state x: 

F{x + ^{x)eB} = P{x,B), BeBiK). 

Let /i„ denote the distribution of X„; then equalities /i„ — /i„_iP and 
A'n = MqP" hold. Formally, define the renewal (or potential) kernel Q by 
the equality 



g(.,.) = £p"(-,-). 



n=0 
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We assume that the Markov chain X is transient (see Meyn and Tweedie 
[T9l Ch. 8]), that is, there exists a countable cover of R with uniformly 
transient sets {-Bfe}. In its turn a set -B € ^(R) is called uniformly transient 
if 

sup(3(y, S) < oo. (3) 

By the Markov property, this is equivalent to 

supQ(y,B) < oo. (4) 

2/eR 

Indeed, considering the first hitting time of _B, we conclude the following 
inequality, for each state cc G R, 

Q{x,B) < supQ(y,B), (5) 

yeB 

which implies Q- 

In the present paper we assume that X^ is transient with respect to the 
collection of sets Bk — {k, k + 1], k G Z; that is, for any fc e Z, 

snpQ{y,{k,k + l]) < oo. (6) 

yen 

Then Q{x, B) < oo for all x and bounded set B. Hence the renewal measure 
generated by the chain X 

oo oo 

U{B) = ^P{X„eB}-^A*„(i?) = (^oQ)(i?) 



is well defined for every initial distribution /iq and bounded set B; U is 
cr-finite with respect to the collection of sets {k,k + I], fc e Z. 

The main goal of our analysis is the local asymptotic behaviour of this 
renewal measure. Without further restrictions on the chain X„, the asymp- 
totics oiU{x,x + h] as a; ^ oo can be very special. We consider a transient 
Markov chain Xn as a perturbation in space of the random walk Sn with 
positive drift. To get similar renewal behaviour for Xn as for 5„, it is natu- 
ral to assume that, being far away from the origin, Xn behaves almost like 

Thus we restrict our attention to the asymptotically space-homogeneous 
Markov chain X, that is, we assume that the distribution of the jump S,(x) 
has a weak limit F as x ^> oo. Let ^ be a random variable with distribution 
F. 

The notion of asymptotically space-homogeneous Markov chain is a nat- 
ural generalisation of both (i) the random walk Sn] in this case £^(x) =d C 
for all x] (ii) the reflected random walk Wn on the positive half-line; in this 
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case ^(x) =d (a: + ^)+ — x. An asymptotically space-homogeneous Markov 
chains appear in different areas; in particular, we are motivated by theory 
of queues when the service rate depends on the current waiting time; and by 
sequential analysis related to an optimal solutions in a change-point prob- 
lem (see Borovkov [4]). Some limit theorems for them were obtained by 
Korshunov |15| . 

So, let the Markov chain X„ be asymptotically space-homogeneous. 

Theorem 1. Let ^{x) ^ ^ as x ^ oo and E^ > 0. Let the family of 
random variables {|f(x)|, x G R} admit an integrahle majorant rj, that is, 
E?7 < oo and 



that 



\^{x)\ <st V forallxeR. (7) 



sup[/(fc,fc+l] < oo. (8) 

fcez 



Assume also that there exists a limit 

po = lim P{X„ > 0}. (9) 

n — >oo 

// the limit distribution F is non-lattice, then U{x,x + h] ^ h/E^ as 
X -^ oo, for every fixed h > 0. 

If the chain Xn is integer valued and Z is the lattice with minimal span 
for distribution F, then U{n} -^ 1/E^ as n ^ oo. 

Condition ([7]) and the dominated convergence theorem imply |^| <st t], 
E|^| < oo and E^(x) — > E^ as x —^ oo; in particular, the chain Xn has an 
asymptotically space-homogeneous drift. 

Only conditions ([8]) and ([9]) of Theorem [1] are not formulated in local 
terms, i.e., in terms of one-step transition probabilities. Below, in Theorem 
[21 we give some simple conditions sufficient for ([5]) . Note that the value of po 
in condition ^ may be very sensitive with respect to the local probabilities. 
It can be illustrated by the following example. Let X„ be a chain valued on 
Z with the following transition probabilities: 

Pi,i+i = 3/4, Pi,i-i = 1/4 for i > 1, 
Pi,i+i = 1/4, Pi,i^i = 3/4 for i < -1, 
Po,i=P, Po-i = l-p, 

where p £ [0, 1]. Given Xq — 0, then pq — po{p) — p is increasing from to 
1 simultaneously with p. 

Since the chain is transient, by (|3|) the convergence iin{K) = P{Xn G 
K} -^ holds as n ^ oo for any compact K. Hence, condition ^ is 
equivalent to the convergence, for every fixed xq, 

P{X„ > xo} -^ Po as n — > 00. (10) 
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Proof of Theorem [T] follows some ideas of the operator approach pro- 
posed by Feller [71 Ch. XI]. First of all, condition ([U allows us to apply 
Helly's Selection Theorem to the family of measures {U{k + ■),k G Z+} 
(see, for example. Theorem 2 in [3 Ch. VIII, Sec. 6]). Hence, there 
exists a sequence of points t^ — > oo such that the sequence of measures 
[/„(•) = U{tn + •) converges weakly to some measure A as n ^ cx). The 
following two lemmas describe properties of A. 

Lemma 1. A weak limit A of the sequence of measures t/(t„ + -) satisfies 
the identity X — X * F. 

Proof. The measure A is non- negative and ct- finite with necessity. Fix 
any smooth function f{x) with a bounded support; let A > be such that 
f{x) = for X ^ [— A, ^]. The weak convergence of measures means the 
convergence of integrals 



fix)U{tn +dx)= I f{x)U{t^ + dx) 
-A 



f{x)X{dx) (11) 



as n ^ oo. On the other hand, due to the equality U — fiQ + UP we have 
the following representation for the left side of pTjl : 

A j*A i*oo 

f{x)no{tn+dx)+ fix) P{tn+y,tn + dx)U{tn+dy). (12) 

'A J-A J —oo 

Since / is bounded and fiQ is finite, 

f{x)fio{tn+dx) < \\f\\ctio[tn-A,tn+A]^0 (13) 



as n ^ oo. The second term in (fT2|) is equal to 



U{tn + dy) / f{x)P{tn + y,t„ + dx). 

I-A 



(14) 



The weak convergence P{t,t + ■) ^ F{-) ast— *-oo implies the convergence 
of the inner integral in (fT4)) : 



f{x)F{dx - y); 



f{x)P{t,,+y,t,, + dx) 



l-A J-A 

here the rate of convergence can be estimated in the following way: 

f-A 



A(n,j/) 



-A 
A 

-A 



/(x)(P(t„ + y,tn + dx) - F{dx ~ y)) 
/'(a;)(P{C(i„ +y)<x-y}-F{x- y))dx 



< II/'IIC / ' \P{atn+y)<x}-F{x)\dx. 

-A-y 



6 Korshunov 

Thus, the asymptotic homogenuity of the chain yields for every fixed C > 
the uniform convergence 

sup A{n,y) -^ as n — > oo. (15) 

yel-C,c] 

In addition, by majorisation condition ([7]), for all x regardless positive or 
negative, 

\P{atn+y)<x}-F{x)\ < 2P{r,>|x|}. 

Hence, for all y, 

A(n,y) < 2||/'||c / \{r,>\x\}dx 

< AA\\f'\\cP{v>\y\-A}. (16) 

We have the estimate 

U{tn + dy)l f fix)Pitn+y, t„+dx) - f f{x)F{dx-y) 

\J—oc J —oo J 

< I A{y,n)U{tr, + dy). 

J —oo 

For any fixed C > 0, uniform convergence (jlSp implies 

/ A{y,n)U{tn + dy) < sup A{y,n) ■ supU[tn ~ C,t„ + C] 

J-C yi^[-C,C] n 

-^ as n — > oo. 
The remaining part of the integral can be estimated by (|16p: 

limsup / A{y,n)U{tn + dy) 

ra^oo J\y\>C 

< 4A||/'||c limsup /" P{ri>\y\-A}Uitn + dy). 

ri^oo J\y\>C 

Since rj has a finite mean, property ^ of the renewal measure U allows us 
to choose a sufficiently large C in order to make the 'limsup' as small as we 
please. Therefore, A„ ^ as n ^ oo. Hence, (|14p has the same limit as 
the sequence of integrals 

U{tn+dy) I f{x)F{dx-y). 
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Now the weak convergence to A implies that (fT4|l has the hmit 

X{dy) I f{x)F{dx ~y) = I f{x) / F{dx - y)X{dy) 

f{x){F*X)idx). (17) 

By ([TTl)-(Il3l) and ^7}, we conclude the identity 

f{x)X{dx) = / f{x){F*X){dx). 



Since this identity holds for every smooth function / with a bounded sup- 
port, the measures A and F * X coincide. The proof is complete. 

Further we use the following statement which was proved in [6] (see also 
[21] or [22 Sec. 5.1]): 

Lemma 2. Let F be a distribution not concentrated at 0. Let X be a 
nonnegative measure satisfying the equality X — F * X and the property 
sup X[n, n + 1] < oo. 

ngZ 

// F is non-lattice, then X is proportional to Lebesgue measure. 
Lf F is lattice with minimal span 1 and A(Z) = 1, then X is proportional 
to the counting measure. 

The concluding part of the proof of Theorem [1] will be carried out for 
the non-lattice case. Choose any sequence of points t„ — > oo such that the 
measure C/(t„ -I- •) converges weakly to some measure A as n — > cx). It follows 
from Lemmas [1] and [2] that then X{dx) — a ■ dx with some a, i.e., 

U{tn + dx) =^ a ■ dx as n ~f cx3. 

Now it suffices to prove that a — po/F,^. 

Fix some keN. Put Uk = UP'' = J^JLk l^r Then 

Uk{tn + dx) ^ a ■ dx as n — > oo. (18) 

Consider the measure Uk — Uk+i = Uk{I — P); by the definition of the 
renewal measure it is equal to /Zfc, that is, for any bounded Borel set B, 
Uk{B) — Uk+i{B) = HkiB) (the equality may fail for unbounded sets, say, 
for (— oo, a;]). In particular, 

(Uk -Uk+i)iO,x] = ^fc(0, x] — > ^fc(0, oo) as a; ^ oo. (19) 

On the other hand, 



(Uk - C/fe+i)(0, x]= (/ - P){y, (0, x])Uk{dy) 

J — OO 

Piy,{0,x])Uk{dy)+ f Piy,{~^,0])Ukidy) 

x: Jo 

' /"OO 

P{y,{x,^))Uk{dy)~ P{y,{0,x])Uk{dy). (20) 
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The asymptotic homogeneity of the chain and weak convergence ((T5)) imply 
the foUowing convergences of the integrals, for any fixed ^ > 0: 

/" P(y,(t„,cx)))C/fe(d2/) -> a / F{C > z}dz (21) 

as n —^ oo, and 

ptn+A pA 

/ P{y,{Q,t,,])Uk{dy) ^ a P{^ < -z}dz. (22) 

Jtn Jo 

Majorisation condition ([7]) allows us to estimate the tails of the integrals: 

tn—A />00 

P(y,(t„,oo))C/fe(dy) < -/ P{rj> z}U{tn-dz) (23) 
Ja 

and 

/•oo /"OC 

/ P(y,(0,t„])C/fc(d2/) < / P{ri>z}U{tn + dz). (24) 

Since the majorant rj is integrable, condition ([H]) guarantees that the right 
sides of inequalities ([23]) and ((24)l can be made as small as we please by the 
choice of sufficiently large A. By these reasons we conclude from ((20)) "p2 |) 
that, as n — > oo, 

{Uk-Uk+i){0,tn] 

Piy,{0,^))Ukidy)+ / P{y,{~^,0])Uk{dy) 

/•OC /"OO 

+« / P{(,>z}dz-a P{£,<-z}dz. 
Jo Jo 

Together with (J19p it implies the following equality, for any fixed k: 

/O /"OO 

P{y,{Q,oo))Uk{dy)+ / ^(y, (-cx3,0])C/fc(dy) + aE^ 

^° (25) 

Now let k ^ oo, then both integrals go to zero. For example, the first 
integral can be estimated in the following way, for every A > 0: 

Piy,iO,^))Ukidy) < I P{ij>-y}U{dy) + Uk{~AM- 

Here, for any fixed A, Uk{~A, 0] ^ as fc — > oo by ^. Therefore, it follows 
from (|25p and ([9]) that po = aE^. The proof of Theorem [T] is complete. 

In the next theorem we provide some simple conditions sufficient for 
condition ([5]), that is, for local compactness of the renewal measure. Denote 
a Ab = min{a,5}. 
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Theorem 2. Suppose that there exists A > such that 

e = inf E(^(a;) A A) > 0. (26) 

xGR 

In addition, let 

S = ini P{Xn> X for aUn>l\Xo = x}>0. (27) 

Then U{x, x + h] < {A + h)/e6 for all x £ R and h > 0; in particular, ([8]) 
holds. 



Proof. Inequality ([5]) implies 



U{x,x + h] = / Q{y,{x,x + h])^ioidy) 

JR 

< sup Q{y,{x,x + h]). 

yfE{x,x-\-h] 

Therefore, it suffices to prove that 

Q{y,{x,x + h]) < {A + h)/eS (28) 

for all y € {x,x + h]. Given Xq G {x,x + h], consider the stopping time 

T = min{n > 1 : Xn > x + h}. 
Since Xr A (x + h + A) - Xq < A + h with probability 1, 
A + h > 'EiXr A{x + h + A)-Xo) 

= Yj E[^" A{x + h + A)- X„_i A{x + h + A)]I{t > n}. 

Hence, the definition of t implies 

oc 

A + /i > ^E{X„ A (x + /i + A) -X„_i A (a; + /i + A);r > n} 

n=\ 

OO 

= ^ E{X„ A (x + ft + A) - X„_i|r > n}P{T > n}. 

n=l 

The Markov property and condition (|26p yield 

E{X„ A(x + ft + A) -X„_i|t > n} > E(C(X„_i) A A) > e 
for all n. Therefore, 



A + h > e^PJT > n} = £Et. 
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So, the expected number of visits to the interval {x, x + h] till the first exit 
from (—00, a; + h] does not exceed {A + h)/e, independently of the initial 
state Xq G {x,x + h]. By condition (|77)) . after exit from (— cx3, a: + h] the 
chain is above the level Xr forever with probability at least S; in particular, 
it does not visit the interval {x, x + h] any more. With probability at most 
1 — S the chain visits this interval again, and so on. Concluding, we get that 
the expected number of visits to the interval (x, x + h] cannot exceed the 
value 

A + h^,^ „_ A + h 



E(i-^) 



n— 

and (|^5|) is proved. The proof of Theorem [2] is complete. 
The latter theorem yields the following 

Corollary 1. Let the family of jumps {^{x)^x £ R} possess an inte- 
grahle m,inorant with a positive mean, that is, there exist a random variable 
C such that EC > and ^(x) >st C for any a; G R. Then 

U{x,x + h] < {A + h)A/e^ 

for any A> Q such that e = E(C /\ A) > 0; in particular, ([5]) holds. 

Proof. Consider the partial sums Z^ — C,i -\' ■ ■ ■ + C,n oi ssa independent 
copies of C,. Denote the first ascending ladder epoch by % = min{n > 1 : 
Zn > 0}. It is well known (see, for example. Theorem 2.3(c) in [31 Ch. 
VIII]) that 

P{Z„ > for all n > 1} = l/Ex- 

Since 

P{Xn> xior alln>l\Xo = x} > P{Z„ > for aU n > 1} 

by the minorisation condition, the S in Thcorcm[2]is at least 1/Ex. Taking 
into account the inequality Ex < A/e, we get 6 > e/A, which implies the 
corollary conclusion. 

If the chain X has a non-negative jumps ^{x) > 0, then the minorisation 
condition is equivalent to the existence of a positive A such that 

7 = inf P{^(a;) > A} > 0. (29) 

In that case one can choose C taking values and A with probabilities 1 — 7 
and 7 respectively; then s > ^A and U{x, a; + /i] < (A + K)I^'^A. 

We conclude with a counterexample demonstrating that condition ([7| 
in Theorem [T] is essential; the existence of integrable majorant cannot be 
relaxed to the condition of the uniform integrability of jumps. We consider 
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an integer valued chain X. For any state k £ Z+, define the transition 
probabihties in the following way: given 2" — 1 < fc < 2"+^ — 2, 

Pk,k = 1/2, 

Pk,k+1 = l/2-pfe^2"+i, 

1 1 

^'=^2"^' " (2»+i - fc) ln(n + e2) " 2' 

The corresponding jumps ^(fc) converge weakly as fc ^ cxd to the Bernoulli 
distribution; they are uniformly integrable. But we can observe a concen- 
tration of relatively large masses at points 2"+^; the renewal measure at 
point 2"+^ is not less than, up to a positive constant, 

2"+i-2 2"+l , 

El 1 Y^ 1 nln2 

(2"+i - fc)ln(n + e2) ^ ln(n + e2) ^ fc ^ Inn' 

Hence, there is no convergence U{k} -^ 1/E^ — 2 and the key renewal 
theorem does not hold for the chain constructed. 

This paper was mostly written while the author was visiting the Boole 
Centre for Research in Informatics, University College Cork, thanks to the 
hospitality of Neil O'Connell and financial support of Science Foundation 
Ireland, grant no. SFI 04/RP1/I512. Also this work was partially supported 
by Russian Science Support Foundation. 
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